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Document Interpreting Systems 

Background of the Invention 

1 . Field of the Invention 

5 This invention relates to document interpreting systems and is 

especially applicable to such system for assisting learning of reading or 
pronunciation or understanding of words, phrases or sentences or 
interpretation of drawings, pictures, etc. It should be understood that the 
term "document" covers any printed matter or indeed any written or drawn 
10 matter, and in particular includes books. 

2. Description of the Related Art 

For effective learning, particularly of language, it is most helpful for a 
student to have the advantage of considerable attention, if not full time 

15 attention, of a teacher. For example, a young child will often learn to read by 
looking at a book containing pictures and corresponding words with an adult 
reading the words as the child follows the words and looks at the associated 
pictures. Thus the mind of the child is focusing on an image depicting 
something, is being given the spoken word and is seeing the written word. 

20 Furthermore, from the context the child is absorbing the word within the 
scope of a grammatical structure and is gathering the meaning, either from 
the pictures if relevant or with the aid of an explanation. At any stage the 
teacher can discuss with the child any word or its meaning or draw to the 
child's attention a similar word and provide explanations and definitions of 

25 grammar and meaning, i.e. syntax and semantics. 
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Similarly, when a child is beginning to read, the adult can monitor the 
reading and provide immediate feedback on a one-to-one basis to maximise 
the rate at which the child gains reading skills and remembers vocabulary. 
The interest and motivation of the child can be maintained at high levels. 

However, apart from a domestic situation, the cost of such teaching 
techniques makes them prohibitive, yet there is a very substantial need for 
such additional support, particularly in the areas of adult illiteracy, foreign 
language teaching and remedial language teaching. 

Furthermore, similar needs for feedback and flexible recapitulation of 
concepts, explanations and facts are needed in many of the areas of 
education including mathematics and other science subjects. The availability 
to a student, of a teacher who can be requested to repeat an explanation, 
elaborate on some point of commentary or deal with any other relevant 
question is an extremely valuable resource in many situations. 

It is known to students with e.g tape recordings of an explanation. 
When such recordings are applied to the process of learning to read, a 
simple system is one in which a tape recording of a printed work is given to a 
student and then student has the opportunity of following the words, 
repeating segments of the tape recording and indeed recording his or her 
own attempts at reading the words and comparing with the tape recordings of 
the tutor. However, such a system is inflexible and hard to operate. 

One published approach to the teaching of language is contained in 
PCT International Publication No. WO 83/02188 (MERIT BOND LIMITED) 
wherein printed text is provided with bar codes associated with at least some 
25 of the text, a manually controlled reading device being used to access the bar 
codes as the user may require, and an electronic processing means used to 
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cause the apparatus to synthesise voice reproduction corresponding to the 
text with which the selected bar code, is associated. ,Such a device has 
" limitations in terms of functions fulfilled and in the special preparation 
required of the printed material, i.e. it can not be used with conventional 
5 books. 

Another published approach, (PCT WO 87/06752) in the teaching of 
language is one in which sets of bar codes are arranged on respective lines 
corresponding to lines of printed text. Each bar code set, when accessed, 
causing a particular storage location from a message store to be accessed to 

10 be reproduced by a speech producer. The limitations of such a system are 
that a book can not be used in its original format, since bar codes have to be 
added or overprinted onto the original document at the expense of text. 
Further, the selection of the bar code set may lead to confusion and small 
children may have difficulty scanning a bar code. 

15 Yet another published approach (PCT WO 90/15402) relates to a 

document interpreting system comprising location detecting means on which 
a plurality of documents to be interpreted may be stacked and for detecting 
through said documents the location of pointing means directed at an area of 
the topmost document of said stack, speech storage means for storing 

20 speech relating to different areas of said documents and speech reproduction 
means for reproducing speech stored in said speech storage means 
corresponding to the area of said topmost document to which said pointing 
means is directed. In the system disclosed, the pointing means takes the 
form of a user's finger. 

25 Another published approach (EP 0 572 466) relates to a document 

interpreting system comprising location detecting means on which a book or 
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other stack of documents may be placed and electronic pointing means 
coupled to said location detecting means and adapted in use to be directed 
at an arbitrary area of the topmost document of the stack of documents, said 
location detecting means being adapted to detect electronically through said 
stack the location of said arbitrary area to cause speech stored in a speech 
storage means associated with said arbitrary area to be reproduced. 

A disadvantage with the document interpreting system of EP 0 572 
466 is that if the stack of documents, for example the pages of a book, 
contain documents with metallic embossing or if the humidity of the stack of 
documents is high then there is considerable distortion in the location 
detection process leading to a shift in the detected position of the pointing 
means which can lead to speech associated with an incorrect arbitrary area 

being reproduced. 

The present invention is directed to a document interpretation system 
which overcomes the above limitations and provides for a practical and useful 
device which eliminates the need for specially printed codes associated with 
the words and phrases of the written material an enables a wide variety of 
normal books to be used including those having metallic embossing or 
documents having high humidity. 

Brief summary of the invention 

According to the present invention there is provide a document 
interpreting system comprising location detection means under which may be 
placed one or more documents and for detecting the location of pointing 
means directed at an area of a topmost document of the one or more 
documents, speech storage means for storing speech relating to different 



5 : 
areas of said one. or more documents, and speech reproduction means for 
reproducing speech stored in said speech storage means corresponding to 
the area of said topmost document to which said pointing means is directed, 
wherein said pointing means comprises electronic pointing means coupled to 
said document interpreting system and adapted in use to be directed at any 
arbitrary area of said topmost document, said location detecting means being 
arranged to detect electronically the location of said arbitrary area for causing 
speech stored in said speech storage means corresponding to the arbitrary 
area of said topmost document to which said pointing means is directed to be 
reproduced, wherein said location detecting means comprises a transparent 
or translucent membrane through which the electronic pointing means is 
directed at the arbitrary area of said topmost document. 

Such a system enables, for example, a user to point to any part of a 
document which is being read and to obtain a spoken version of any text e.g. 
words, phrases, sentences, etc in the vicinity of a pointer or a spoken 
description of any picture in said area. It would also be possible for the 
spoken version or description to be in a foreign language thereby assisting 
foreign language learning. It is to be noted that the document being read may 
be in its original form and does not require any additional matter, e.g. bar 
codes for its interpretation. 

An exemplary embodiment of the invention will now be described with 
reference being made to the accompanying drawings. 

Brief Description of the Several Views of the Drawings 

Figure 1 shows a diagrammatic representation of a document 
interpretation system in accordance with the present invention; 
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Figure 2 depicts in greater detail the appearance of a topmost card 
shown in Figure 1 which is useful in explaining the operation of the system of 
Figure 1; 

Figure 3 shows a block schematic diagram of the document 
interpretation system of Figure 7; and 

Figure 4 is a flow diagram relating to the document interpretation 

system of Figures 1 and 3. 

Written Description of the Best Mode for Carrying Out the Invention 
Figure 1 

The document interpretation system depicted in Figure 1 of the 
drawings comprises a membrane 101 which overlies a cartridge 102 in which 
are located one or more cards 103 in the form of a stack. The card to be read 
by the user (not shown) is placed as the topmost card 104 where more than 
one card is present. The cartridge 102 and membrane 101 are shown as 
schematically in Figure 1. In practise, the membrane 101 may be housed in a 
frame formed" as part of the upper surface of the cartridge 102 of fixedly 
located on, or hinged to the upper surface of the cartridge 102 by any 
suitable means. The important aspect is that the membrane 101 is located in 
use, or may be brought into location for use, such that it overlies the card or 
cards 1 03 stored in the cartridge 1 02. 

The membrane 101 is transparent or at least sufficiently translucent as 
to allow the user to determine the contents of the topmost card 104 when 
viewing through the membrane 101. The membrane 101 comprises a grid 
105* formed of conductive stands creating an X-Y pattern dividing the 
membrane 101 into substantially squared shaped segments. 
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The membrane 101 is formed from a pair of transparent films, one of 
which defines a set of parallel conductive strands in a first (X) direction, and 

the second film defining conductive strands in a second (Y) direction 

«■ 

perpendicular to the first direction. The transparent films are Orgacon 

Conductive Transparent Films and the conductive strands are defined by 

- 

screen printing of Strupas ink. The Orgacon films and Strumas ink are 
supplied by Agfa-Gevaert N.V., of Mortsel, Belgium. 

The membrane 101, and specifically the grid 105, is electrically 
coupled to a printed circuit board (PCB) 109. The PCB 109 has mounted 
thereto an Application Specific Integrated Circuit (ASIC) 111. The PCB 109 is 
further provided with a power source in the form of a battery 110 and a 
speaker 112. The PCB 109 may be formed as part of the cartridge 102, as 
part of the frame housing the membrane 101 , or as part of a separate unit. 

The ASIC 111 is provided with its own power source in the form of a 
battery 108. Alternatively, the ASIC 111 may be powered directly from the 
PCB battery 110. 

A pen or pointer 106 is coupled to the PCB 109 by means of a cable 
107. The pointer 106 is adapted to be pointed at an area of the membrane 
101 coincident with a arbitrary area of the topmost card 104 viewed through 
the membrane 101 so as to enable the co-ordinate location of the arbitrary 
area to be detected by the membrane 101 and in turn determined by the 
ASIC 111. 

The pointer 106 is, for example, magnetically or capacitively coupled 
to the membrane 101 in use. Importantly, since the relative positions of the 
membrane 101 and the pointer 106 are determined magnetically or 
capavitively rather than by pressure applied to the membrane 101, the 
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document interpretation system finds particular application for use with young 
children who are apt to press randomly and unintentionally on the membrane 
101 whilst using the system. With a membrane activated by pressure such 
contacts would lead to a large number of false readings and hence confusion 
to the user. 

The ASIC 111 contains machine code and/or software programs and 
solid state circuitry to control operation of the pointer 106, membrane- 101 
and speaker 112. In addition, the ASIC 111 stores speech associated with 

areas of the cards 103. 

In use, the output from the grid 105 of the membrane 101, consisting 
of the co-ordinate details of an area of the card 104 to which the pointer 106 
is directed, is fed to the ASIC 111 on the PCB 109. The machine code and/or 
other software encoded in the ASIC 111 resolves the co-ordinate details from 
the membrane 101 and determines the speech stored in the ASIC 111 which 
is associated with the area of the topmost card 104 to which the pointer 106 
is directed. 

It will be appreciated by those skilled in the art that the location of the 
card 104 or other document to be read, relative to the membrane 101, may 
be established by, for instance, physical means such as the internal 
dimensions of the cartridge 102 to ensure the card is placed in a known 
location relative to the overlying membrane 101. Alternatively, the card 104 or 
other documents may be placed in the cartridge 102 in any orientation and 
known features such as comers of the card 104 may be registered by 
pointing at them through the membrane 101 with the pointer 106 in an 
initialising registration procedure. By this method the location of the words 
and pictures, etc. may be calculated by such known means as simple co- 
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ordinate geometry routines. 

In addition, it will be appreciated that the ASIC 111 will need to 
determine which of the cards 103 is the topmost card 104. This may be done, 
for example, by providing a unique identifier area on each card 103 to which 
5 the pointer 106 is first directed. By arranging for each card 103 in the stack to 
have its identifier area in a. different position, the ASIC 111 is able to 
determine the identity of the topmost card 1 04/ Alternatively, a manual 
mechanism may be provided wherein the PCB 109 is provided with an input 
terminal by which the user may enter an identifier, e.g and alpha-numeric 

10 character or characters, to identify the topmost card 1 04. 

In use of the system thus far described, a user points the pointer 106 
at an area at the topmost card of the stack of cards 103, e.g some printed 
text or a picture. The grid 105 of the membrane 101 detects the presence of 
the pointer 106 by means of the magnetic or capacitive changes in the grid 

15 105 of the membrane 101 caused by the proximity of the pointer 106. The 
membrane 1 01 outputs signals representing the co-ordinates of. the 
designated area to the ASIC 111. The machine code or software program 
embedded in the ASIC 111 processes the signals and resolves the 
corresponding co-ordinates and generates a signal instructing the speaker 

20 112 to output an audio signal corresponding to speech corresponding to the 
printed text pointed to on the card 104 or alternatively corresponding to the : 
picture pointed to on the card 104. 

Optionally, the user of the system can access a button or switch either 
on, for example the stylus to give a real voice reply of individual words when 

25 touched by the pointer 106. Alternatively, another button may be provided 
which when accessed causes reproduction of the whole sentence when the 
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first or any word in that sentence is touched by the pointer 106. Another 
button may be provided which when accessed allows the user to receive 
prompts, such as a suggestion to look the syllables within the word selected 
or the system can be set up to sound out the word phoneme by phoneme. A 
further button may be provided which, when accessed by the user, will give 
an explanation of the word or picture which may be in terms of its definition, 
mode of usage or other commentary. 

The system may also be used for teaching a second language, in 
which case, for example, a sentence can be reproduced in both languages in 
the correct idiom for each language using two different buttons, e.g on the 
cartridge 102. Each word can also be represented in each language, word for 
word, by the selection an appropriate button. Where complex multi-syllable 
foreign words are being taught, the word may be represented both in its 
normal presentation at normal dictation speed and also by a further selection 
at a slowed down speed enabling the student to dissect the pronunciation of 
the word, syllable by syllable. 

Some examples of the use of the system described with reference to 
Figure 1 will now be described with reference to Figure 2 which depicts the 
topmost card 104 of Figure 1 in greater detail. 

Referring to Figure 2, a rectangular area 201 is reserved on cards 103 
for displaying a unique identifier. Thus card 104 has a unique identifier 202 
within the area 201 : Consequently, using pointer 106 of Figure 1 to point to 
rectangular area 202 identifies card 104 to the document interpretation 

system of Figure 1. 

Using pointer 106 to point to area 203 would cause the sentence U K is 
for Katie kangaroo" to be reproduced. Pointing to area 204 may cause the 
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phrase "hello I am Katie kangaroo" to be reproduced. 

Pointing in each of the areas 205 to 213 inclusive would cause the 
individual words "kangaroo", "kitten"; "Kite", "Orange", "K", "is", "for", "Katie" 
and "Kangaroo", respectively, to be reproduced. 

It should be appreciated that the rectangular areas referred to in 
Figure 2 are used by a way of example only and other shaped areas, such as 
a circular shaped areas may be used. 

In Figure 3 of the drawings, there is shown a block schematic diagram 
relating to the document interpretation system of Figure 1. This comprises the 
membrane 101 which affords and output 308 to the PCB 109, which is itself 
connected to the pointer 106 via the cable 107 and to the speaker 112. , 

The PCB 109 has mounted thereon the ASIC 111 which typically 
comprises a microprocessor 330 which operates under the control of a 
program stored in read only memory "ROM" 331 and receives the signals 
representing the co-ordinate details via output 308 from the membrane,101. 
A random access memory (RAM) 332 is provided in which is stored the 
speech signals corresponding to the various areas of the documents being 
read, e.g the cards 103. The micro processor 330 outputs the speech signals 
to the speaker 112, via an input/output (I/O) interface 333 which causes the 
speech signals to be reproduced by the speaker 112. 

It is envisaged that- that random access memory 332 may take the 
form of a removable RAM-card in which the various speech signals are 
stored corresponding to the different areas if the cards 103, thereby enabling 
a set of cards 103 and associated RAM-card to be sold as a package. 

In Figure 4 of the drawings there is shown a flow diagram of a typical 
operating system of the arrangement depicted in Figure 3. At step 401, the 
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micro-processor 330 monitors the membrane 101 for new data. The new 
data is decoded at step 402 to obtain the corresponding speech signal stored 
in the RAM 332. At step 403, the speech signal is output to the speaker 112, 
the output of which is monitored by the micro-processor 330 via the I/O 
5 interface 333, at step 404, to determine the conclusion of the speech signals 
corresponding to the selected area. The system is then ready for re use. 

It will be appreciated that new cards, or locally generated documents, 
<' e.g. flash cards and picture stories, may have their textual and pictorial co- 

ordinates recorded by means similar to that of Figure 1 with the system 

10 operating in a 'Writing" mode. In this mode the co-ordinates of individual 
words, pictures, phrases, etc. are read, via the pointer 106, from cards placed 
in the cartridge 102 under the membrane 101 by the program in the read only 
memory 331 (Figure 3). These co-ordinates are then converted into the 
corresponding speech signals stored in the random access memory 332 for 

15 subsequent use in the "reading" mode described above. 

The document interpreting system which has been described is 
advantageous in that it enables normal printed or written documents to be 
"read" without special overprinting of bar codes, magnetic codes, etc. The 
system can also operate with documents containing metallic embossing or 

20 having pages with high humidity content. RAM-cards and sets of cards 103 
may also be sold as a package in respect to particular items for use by 
teachers, pupils, etc. on equipment in many different locations. 

It should be appreciated that the document interpreting system which 
has been described has been given by a way of example only and various 

25 modifications may be made dependent upon any particular application. 
Although the pointer 106 has been shown as connected to the PCB 109 by 
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. means of a cable 107, it may be remotely connected to it using, for example, 
an infrared, radio or other link. Also, although the use of an ASIC 111 in 
conjunction with a speaker has been described, other speech storage 
systems may be used, such as using tape or compact disk and associated 
5 player. Other output devices may also be used such as headphones. 

Further, whilst the set of cards 103 has been described as being 
stored in a cartridge 102, other means of locating the cards 103 relative to 
the membrane 101 may be utilised, for example, by clipping or otherwise 
temporarily fixing the card 104 or cards 103 to the underside of the 
10 membrane 101. 

Further, a mechanism may be provided to enable the cards 103 in the 
cartridge 104 to be shuffled so as to change the identity of the topmost card 
104. For example, an aperture may be provided in the cartridge 102 to allow 
the stack of cards 103 to be removed, manually shuffled and reinserted into 
15 the cartridge 102. 

The system may also be used with documents other than individual 
cards 103. For example, the system may be used with complete books 
wherein the membrane 101 is placed over each page of the book in turn. 
Advantageously, means may be provided, such as a book stand, for holding 
20 the book at rest with the membrane 101 hinged thereto such that the 
membrane 101 may be hinged upwards away from contact with the page of 
the book to allow for the pages to be turned. The membrane 101 may then 
be^hinged back down into contact with the subsequent page of the book. 

Another form of document that may be used with the system is a scroll 
25 comprising an elongate document wound onto one or a pair of rollers. In use, 
the roller or rollers can be rotated so as to bring the required portion of the 
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document into line with the membrane 101. 

Further, while it is important that the membrane 1 01 is transparent or 
at least translucent, the membrane 101 may be manufactured to be flexible 
or inflexible. In particular, where the membrane 101 is flexible, a separate 
strengthening member may be provided, for example, a frame to stiffen the 
membrane 101. 



Claims 
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1. A document interpreting system comprising location detecting 
means under which may be placed one or more documents and for 
detecting the location of pointing means directed at an area of a topmost 
document of the one or more documents, speech storage means for storing 
speech relating to different areas of said one or more documents, and 
speech reproduction means for reproducing speech stored in said speech 
storage means corresponding to the area of said topmost document to 
which said pointing means is directed, wherein said pointing means 
comprises electronic pointing means coupled to said document interpreting 
system and adapted in use to be directed at an arbitrary area of said 
topmost document, said location detecting means being arranged to detect 
electronically the location of said arbitrary area for causing speech stored in 
said speech storage means corresponding to the arbitrary area of said 
topmost document to which said pointing means is directed to be 
reproduced, wherein said location detecting means comprises a 
transparent or translucent membrane through which the electronic pointing 
means is directed at the arbitrary area of said topmost document. 

2. A document interpreting system as claimed in claim 1, 
wherein the membrane comprises co-ordinate detecting means for 
affording an output corresponding to the co-ordinate location of said 
electronic pointing means on said topmost document. 
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3. A document interpreting system as claimed in claim 2, 
wherein said co-ordinate detecting means comprises a network of 
conductive strands embedded in or in contact with said membrane. 

4. A document interpreting system as claimed in claim 3, 
wherein said conductive strands comprise of a conductive coating on said 
membrane. 

5. A document interpreting system as claimed in any preceding 
claim wherein said electronic pointing means is coupled magnetically to 
said membrane. 

6. A document interpreting system as claimed in any of claims 1 
to 4, wherein said electronic pointing means is coupled capacitively to said 
membrane. 

7. A document interpreting system as claimed in any preceding 
claim, wherein said membrane is flexible. 

8. A document interpreting system as claimed in any of claims 1 
to 7, wherein said membrane is substantially rigid. 

9. A document interpreting system as claimed in any preceding 
claim, wherein said membrane is held in a frame. 
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10. A document interpreting system as claimed in any preceding 
claim, wherein said speech storage means comprises an abdication specific 
integrated circuit (ASIC) and said speech reproduction means comprises a 
speaker. 

5 

11. A document interpreting system as claimed in claim 10, 
wherein said ASIC comprises a random access memory (RAM) for storing 
speech signals corresponding to areas of said one or more documents, a 
micro-processor which operates under the control of a program stored in a 

10 read only memory (ROM) to decode signals received from the membrane 
indicative of the location of said arbitrary area and to instruct said random 
access memory to output one or more of said speech signals corresponding 
to said arbitrary area to said speaker, via an input/output interface to cause 
said one or more speech signals to be reproduced by the speaker. 

15 

12. A document interpreting system as claim in any preceding 
claim, wherein said one ore more documents are received in use in a 
housing for maintaining said documents in a stack with one of said 
documents topmost so as to be visible through said membrane. 

20 

13. A document interpreting system as claimed in claim 12, 
wherein said housing comprises means for shuffling said one or more 
documents so as to change the identity of said topmost document. 

25 14. A document interpreting system as claimed in any preceding 

claim, wherein said documents are individual cards. 
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15. A document interpreting system as claimed in any preceding 
claim, wherein said one or more documents comprise an elongate sheet 
wound onto one or more rollers. 

16. A document interpreting system substantially as herein before 
described with reference to or as shown in the accompanying drawings. 
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